Cross-identity Video Motion Retargeting with Joint Transformation and Synthesis

In this paper, we propose a novel dual-branch Transformation-Synthesis network (TS-Net), for video motion retargeting. Given one subject video and one driving video, TS-Net can produce a new plausible video with the subject appearance of the subject video and motion pattern of the driving video. TS-Net consists of a warp-based transformation branch and a warp-free synthesis branch. The novel design of dual branches combines the strengths of deformation-grid-based transformation and warp-free generation for better identity preservation and robustness to occlusion in the synthesized videos. A mask-aware similarity module is further introduced to the transformation branch to reduce computational overhead. Experimental results on face and dance datasets show that TS-Net achieves better performance in video motion retargeting than several state-of-the-art models as well as its single-branch variants. Our code is available at https://github.com/nihaomiao/WACV23_TSNet.Comment: WACV 202

NeRF-Enhanced Outpainting for Faithful Field-of-View Extrapolation

In various applications, such as robotic navigation and remote visual assistance, expanding the field of view (FOV) of the camera proves beneficial for enhancing environmental perception. Unlike image outpainting techniques aimed solely at generating aesthetically pleasing visuals, these applications demand an extended view that faithfully represents the scene. To achieve this, we formulate a new problem of faithful FOV extrapolation that utilizes a set of pre-captured images as prior knowledge of the scene. To address this problem, we present a simple yet effective solution called NeRF-Enhanced Outpainting (NEO) that uses extended-FOV images generated through NeRF to train a scene-specific image outpainting model. To assess the performance of NEO, we conduct comprehensive evaluations on three photorealistic datasets and one real-world dataset. Extensive experiments on the benchmark datasets showcase the robustness and potential of our method in addressing this challenge. We believe our work lays a strong foundation for future exploration within the research community

Development of anti-infectives using phage display: biological agents against bacteria, viruses, and parasites

The vast majority of anti-infective therapeutics on the market or in development are small molecules; however, there is now a nascent pipeline of biological agents in development. Until recently, phage display technologies were used mainly to produce monoclonal antibodies (MAbs) targeted against cancer or inflammatory disease targets. Patent disputes impeded broad use of these methods and contributed to the dearth of candidates in the clinic during the 1990s. Today, however, phage display is recognized as a powerful tool for selecting novel peptides and antibodies that can bind to a wide range of antigens, ranging from whole cells to proteins and lipid targets. In this review, we highlight research that exploits phage display technology as a means of discovering novel therapeutics against infectious diseases, with a focus on antimicrobial peptides and antibodies in clinical or preclinical development. We discuss the different strategies and methods used to derive, select, and develop anti-infectives from phage display libraries and then highlight case studies of drug candidates in the process of development and commercialization. Advances in screening, manufacturing, and humanization technologies now mean that phage display can make a significant contribution in the fight against clinically important pathogens

Synthetic Augmentation with Large-scale Unconditional Pre-training

Deep learning based medical image recognition systems often require a substantial amount of training data with expert annotations, which can be expensive and time-consuming to obtain. Recently, synthetic augmentation techniques have been proposed to mitigate the issue by generating realistic images conditioned on class labels. However, the effectiveness of these methods heavily depends on the representation capability of the trained generative model, which cannot be guaranteed without sufficient labeled training data. To further reduce the dependency on annotated data, we propose a synthetic augmentation method called HistoDiffusion, which can be pre-trained on large-scale unlabeled datasets and later applied to a small-scale labeled dataset for augmented training. In particular, we train a latent diffusion model (LDM) on diverse unlabeled datasets to learn common features and generate realistic images without conditional inputs. Then, we fine-tune the model with classifier guidance in latent space on an unseen labeled dataset so that the model can synthesize images of specific categories. Additionally, we adopt a selective mechanism to only add synthetic samples with high confidence of matching to target labels. We evaluate our proposed method by pre-training on three histopathology datasets and testing on a histopathology dataset of colorectal cancer (CRC) excluded from the pre-training datasets. With HistoDiffusion augmentation, the classification accuracy of a backbone classifier is remarkably improved by 6.4% using a small set of the original labels. Our code is available at https://github.com/karenyyy/HistoDiffAug.Comment: MICCAI 202

Conditional Image-to-Video Generation with Latent Flow Diffusion Models

Conditional image-to-video (cI2V) generation aims to synthesize a new plausible video starting from an image (e.g., a person's face) and a condition (e.g., an action class label like smile). The key challenge of the cI2V task lies in the simultaneous generation of realistic spatial appearance and temporal dynamics corresponding to the given image and condition. In this paper, we propose an approach for cI2V using novel latent flow diffusion models (LFDM) that synthesize an optical flow sequence in the latent space based on the given condition to warp the given image. Compared to previous direct-synthesis-based works, our proposed LFDM can better synthesize spatial details and temporal motion by fully utilizing the spatial content of the given image and warping it in the latent space according to the generated temporally-coherent flow. The training of LFDM consists of two separate stages: (1) an unsupervised learning stage to train a latent flow auto-encoder for spatial content generation, including a flow predictor to estimate latent flow between pairs of video frames, and (2) a conditional learning stage to train a 3D-UNet-based diffusion model (DM) for temporal latent flow generation. Unlike previous DMs operating in pixel space or latent feature space that couples spatial and temporal information, the DM in our LFDM only needs to learn a low-dimensional latent flow space for motion generation, thus being more computationally efficient. We conduct comprehensive experiments on multiple datasets, where LFDM consistently outperforms prior arts. Furthermore, we show that LFDM can be easily adapted to new domains by simply finetuning the image decoder. Our code is available at https://github.com/nihaomiao/CVPR23_LFDM.Comment: CVPR 202

The Grism Lens-amplified Survey from Space (GLASS). X. Sub-kiloparsec Resolution Gas-phase Metallicity Maps at Cosmic Noon behind the Hubble Frontier Fields Cluster MACS1149.6+2223

We combine deep Hubble Space Telescope grism spectroscopy with a new Bayesian method to derive maps of gas-phase metallicity for 10 star-forming galaxies at high redshift ($1.2\lesssim z\lesssim 2.3$). Exploiting lensing magnification by the foreground cluster MACS1149.6+2223, we reach sub-kiloparsec spatial resolution and push the limit of stellar mass associated with such high-z spatially resolved measurements below ${10}^{8}\,{M}_{\odot }$ for the first time. Our maps exhibit diverse morphologies, indicative of various effects such as efficient radial mixing from tidal torques, rapid accretion of low-metallicity gas, and other physical processes that can affect the gas and metallicity distributions in individual galaxies. Based upon an exhaustive sample of all existing sub-kiloparesec resolution metallicity gradient measurements at high z, we find that predictions given by analytical chemical evolution models assuming a relatively extended star-formation profile in the early disk-formation phase can explain the majority of observed metallicity gradients, without involving galactic feedback or radial outflows. We observe a tentative correlation between stellar mass and metallicity gradients, consistent with the "downsizing" galaxy formation picture that more massive galaxies are more evolved into a later phase of disk growth, where they experience more coherent mass assembly at all radii and thus show shallower metallicity gradients. In addition to the spatially resolved analysis, we compile a sample of homogeneously cross-calibrated integrated metallicity measurements spanning three orders of magnitude in stellar mass at z ~ 1.8. We use this sample to study the mass–metallicity relation (MZR) and find that the slope of the observed MZR can rule out the momentum-driven wind model at a 3σ confidence level

The Grism Lens-amplified Survey from Space (GLASS). X. Sub-kiloparsec Resolution Gas-phase Metallicity Maps at Cosmic Noon behind the Hubble Frontier Fields Cluster MACS1149.6+2223

We combine deep Hubble Space Telescope grism spectroscopy with a new Bayesian method to derive maps of gas-phase metallicity for 10 star-forming galaxies at high redshift ($1.2\lesssim z\lesssim 2.3$). Exploiting lensing magnification by the foreground cluster MACS1149.6+2223, we reach sub-kiloparsec spatial resolution and push the limit of stellar mass associated with such high-z spatially resolved measurements below ${10}^{8}\,{M}_{\odot }$ for the first time. Our maps exhibit diverse morphologies, indicative of various effects such as efficient radial mixing from tidal torques, rapid accretion of low-metallicity gas, and other physical processes that can affect the gas and metallicity distributions in individual galaxies. Based upon an exhaustive sample of all existing sub-kiloparesec resolution metallicity gradient measurements at high z, we find that predictions given by analytical chemical evolution models assuming a relatively extended star-formation profile in the early disk-formation phase can explain the majority of observed metallicity gradients, without involving galactic feedback or radial outflows. We observe a tentative correlation between stellar mass and metallicity gradients, consistent with the "downsizing" galaxy formation picture that more massive galaxies are more evolved into a later phase of disk growth, where they experience more coherent mass assembly at all radii and thus show shallower metallicity gradients. In addition to the spatially resolved analysis, we compile a sample of homogeneously cross-calibrated integrated metallicity measurements spanning three orders of magnitude in stellar mass at z ~ 1.8. We use this sample to study the mass–metallicity relation (MZR) and find that the slope of the observed MZR can rule out the momentum-driven wind model at a 3σ confidence level

Hierarchical colour image segmentation by leveraging RGB channels independently

In this paper, we introduce a hierarchical colour image segmentation based on cuboid partitioning using simple statistical features of the pixel intensities in the RGB channels. Estimating the difference between any two colours is a challenging task. As most of the colour models are not perceptually uniform, investigation of an alternative strategy is highly demanding. To address this issue, for our proposed technique, we present a new concept for colour distance measure based on the inconsistency of pixel intensities of an image which is more compliant to human perception. Constructing a reliable set of superpixels from an image is fundamental for further merging. As cuboid partitioning is a superior candidate to produce superpixels, we use the agglomerative merging to yield the final segmentation results exploiting the outcome of our proposed cuboid partitioning. The proposed cuboid segmentation based algorithm significantly outperforms not only the quadtree-based segmentation but also existing state-of-the-art segmentation algorithms in terms of quality of segmentation for the benchmark datasets used in image segmentation. © 2019, Springer Nature Switzerland AG

A Collage of Small Planets from the Lick-Carnegie Exoplanet Survey: Exploring the Super-Earth and Sub-Neptune Mass Regime

Analysis of new precision radial velocity (RV) measurements from the Lick Automated Planet Finder and Keck HIRES has yielded the discovery of three new exoplanet candidates orbiting the nearby stars HD 190007 and HD 216520. We also report new velocities from the APF and the Planet Finder Spectrograph and updated orbital fits for the known exoplanet host stars GJ 686 and HD 180617. Of the newly discovered planets, HD 190007 b has a period of P = 11.72 days, an RV semiamplitude of K = 5.64 ± 0.55 m s-1, a minimum mass of M pl = 16.46 ± 1.66 M ⊕, and orbits the slightly metal-rich, active K4V star HD 190007. For HD 216520 b, we find P = 35.45 days, K = 2.28 ± 0.20 m s-1, and M pl = 10.26 ± 0.99 M ⊕, while for HD 216520 c, P = 154.43 days, K = 1.29 ± 0.22 m s-1, and M pl = 9.44 ± 1.63 M ⊕. Both planets orbit the slightly metal-poor, inactive K0V star HD 216520. Our updated best-fit models for HD 180617 b and GJ 686 b are in good agreement with the published results. For HD 180617 b, we obtain P = 105.91 days and M pl = 12.214 ± 1.05 M ⊕. For GJ 686 b, we find P = 15.53 days and M pl = 6.624 ± 0.432 M ⊕. Using an injection-recovery exercise, we find that HD 190007 b and HD 216520 b are unlikely to have additional planets with masses and orbital periods within a factor of 2, in marked contrast to ∼85% of planets in this mass and period range discovered by Kepler

A Robust, Simple Genotyping-by-Sequencing (GBS) Approach for High Diversity Species

Advances in next generation technologies have driven the costs of DNA sequencing down to the point that genotyping-by-sequencing (GBS) is now feasible for high diversity, large genome species. Here, we report a procedure for constructing GBS libraries based on reducing genome complexity with restriction enzymes (REs). This approach is simple, quick, extremely specific, highly reproducible, and may reach important regions of the genome that are inaccessible to sequence capture approaches. By using methylation-sensitive REs, repetitive regions of genomes can be avoided and lower copy regions targeted with two to three fold higher efficiency. This tremendously simplifies computationally challenging alignment problems in species with high levels of genetic diversity. The GBS procedure is demonstrated with maize (IBM) and barley (Oregon Wolfe Barley) recombinant inbred populations where roughly 200,000 and 25,000 sequence tags were mapped, respectively. An advantage in species like barley that lack a complete genome sequence is that a reference map need only be developed around the restriction sites, and this can be done in the process of sample genotyping. In such cases, the consensus of the read clusters across the sequence tagged sites becomes the reference. Alternatively, for kinship analyses in the absence of a reference genome, the sequence tags can simply be treated as dominant markers. Future application of GBS to breeding, conservation, and global species and population surveys may allow plant breeders to conduct genomic selection on a novel germplasm or species without first having to develop any prior molecular tools, or conservation biologists to determine population structure without prior knowledge of the genome or diversity in the species